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Abstract 

In  this  paper,  we  consider  how  eigenspaces  of  a  Hermitian  matrix  A 
change  when  it  is  perturbed  to  A  =  D* AD  and  how  singular  values  of 
a  (nonsquare)  matrix  B  change  when  it  is  perturbed  to  B  =  DIBD2, 
where  D,  D 1  and  D2  are  assumed  to  be  close  to  identity  matrices  of 
suitable  dimensions,  or  either  D 1  or  D2  close  to  some  unitary  matrix.  We 
have  been  able  to  generalize  well-known  Davis-Kahan  sin  9  theorems.  As 
applications,  we  obtained  bounds  for  perturbations  of  graded  matrices. 


We  will  follow  the  notation  introduced  in  Li  [7]. 


1  Introduction 

Let  A  and  A  be  two  n  x  n  Hermitian  matrices  with  eigendecompositions 

A  =  UAU*  =  (UUU2)  (  Al  a2  )  (  u\  )  ’  V1) 
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under  grant  No.  20552402  and  the  University  of  Tennessee  through  the  Advanced  Research 
Projects  Agency  under  contract  No.  DAAL03-91-C-0047,  by  the  National  Science  Foundation 
under  grant  No.  ASC-9005933,  and  by  the  National  Science  Infrastructure  grants  No.  CDA- 
8722788  and  CDA-9401156. 


1 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

25  JUL  1994 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-1994  to  00-00-1994 

4.  TITLE  AND  SUBTITLE 

Relative  Perturbation  Theory:  (II)  Eigenspace  Variations 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

University  of  California  at  Berkeley, Department  of  Electrical 

Engineering  and  Computer  Sciences, Berkeley, CA, 94720 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

In  this  paper,  we  consider  how  eigenspaces  of  a  Hermitian  matrix  A  change  when  it  is  perturbed  to  ~  A  =  D 
*  AD  and  how  singular  values  of  a  (nonsquare)  matrix  B  change  when  it  is  perturbed  to  ~  B  =  D1  BD2, 
where  D,  D1  and  D2  are  assumed  to  be  close  to  identity  matrices  of  suitable  dimensions,  or  either  D1  or  D2 
close  to  some  unitary  matrix.  We  have  been  able  to  generalize  well-known  Davis-Kahan  sin  theta  theorems. 
As  applications,  we  obtained  bounds  for  perturbations  of  graded  matrices. 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

35 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


(1.2) 


A  =  UAU*  =  (UUU2)  ^  Al  ~r 

where  U,  U  £  UTn,  Ui,  Ui  £  £.nxk  (1  <  k  <  n)  and 

Ai  =  diag(A1;  •  •  •,  A*),  A2  =  diag(Afc+1,  •  •  • ,  A„),  (1.3) 

Ai  =  diag(Ai,  •  •  • ,  Xk),  A2  =  diag(Afc+i,  •  •  • ,  A„).  (1.4) 

Suppose  now  that  A  and  A  are  close.  The  question  is:  How  close  are  the 

eigenspaces  spanned  by  Ui  and  Ui  ?  This  question  has  been  well  answered  by 

four  celebrated  theorems  so-called  sin!?,  tan  9,  8^20  and  tan  26  due  to  Davis 
and  Kahan  [2]  for  arbitrary  perturbations  in  the  sense  that  the  perturbations 
to  A  can  be  made  arbitrary  as  long  as  A  —  A  is  kept  small.  This  paper,  on  the 
other  hand,  will  address  the  following  question:  How  close  are  the  eigenspaces 
spanned  by  Ui  and  Ui  under  the  assumption  that  A  =  D*  AD  for  some  D  close 
to  I?  A  similar  question  for  singular  value  decompositions  will  be  answered 
also.  We  will  deal  with  perturbations  of  the  following  kinds: 

•  Eigenvalue  problems: 

1.  A  and  A  =  D* AD  for  Hermitian  case,  where  D  is  nonsingular  and 
close  to  I  or  more  generally  to  a  unitary  matrix; 

2.  H  =  D*  AD  and  H  =  D*  AD  for  graded  nonnegative  Hermitian  case, 
where  it  is  assumed  that  A  and  A  are  nonsingular  and  often  that  D 
is  a  highly  graded  diagonal  matrix  (this  assumption  is  not  necessary 
to  our  theorems  below). 

•  Singular  value  problems: 

1.  B  and  B  =  D\BD 2,  where  D\  and  D2  are  nonsingular  and  close  to 
I  or  more  generally  to  two  unitary  matrices; 

2.  G  =  BD  and  G  =  BD  for  graded  case,  where  it  is  assumed  that  B 
and  B  are  nonsingular  and  often  that  D  is  a  highly  graded  diagonal 
matrix  (this  assumption  is  not  necessary  to  our  theorems  below). 

Recently,  Eisenstat  and  Ipsen  [4]  launched  an  attack  towards  the  above  men¬ 
tioned  perturbations  except  graded  cases.  We  will  give  a  brief  comparison  among 
their  results  and  ours. 

Notation:  For  X  £  Gnxk,  7Z(X)  denotes  the  subspace  spanned  by  the  column 
vectors  of  X . 

Let  both  X,  Y  £  Cnxk  (n  >  k)  have  full  column  rank  k,  and  define  the 
angle  matrix  0(X,  Y)  between  X  and  Y  as 

0(X,  Y)  d=  arccos ((X* X)-^X*Y(Y*Y)~1Y*X(X* X)~^)~X 
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The  canonical  angles  between  the  subspace  X  =  1Z(X)  and  y  =  1Z(Y)  are 
dehned  to  be  the  singular  values  of  the  Hermitian  matrix  0(X,  Y ) .  The  following 
lemma  is  well-known.  For  a  proof  of  it,  the  reader  is  referred  to,  e.g,  Li  [6, 
Lemma  2.1]. 

Lemma  1.1  Suppose  that  Y  =  (Y,  Y\)  £  Cnxn  is  a  nonsingular  matrix  with 
Y-1  =  (^Ssl^j  ,  S  £  Cnxk. 

Then  for  any  unitarily  invariant  norm  |||  •  ||| 


|sm0(X,Y)|||  = 


Taking  X  =  U\  and  X  =  U\  (Ref.  (1.1)  and  (1.2)),  one  has 


Q{Ui,Ui)  =  arccos(Y1*Yii71*i7i)-Y 


sin0(17i,  U\) 


u*2ia 


(1.5) 


For  more  discussions  on  angles  between  subspaces,  the  reader  is  referred  to  Davis 
and  Kahan  [2]  and  Stewart  and  Sun  [9,  Chapters  I  and  II]. 

This  paper  is  organized  as  follows.  We  briefly  review  Davis-Kahan  sin  9 
theorems  and  their  generalizations — Wedin  sin  9  theorems — to  singular  value 
decompositions  in  §2.  In  §3,  we  first  define  four  indispensable  relative  gap 
functions.  We  present  our  sin#  theorems  for  A  and  A  =  D* AD  in  §3.2,  those 
for  graded  nonnegative  Hermitian  matrices  in  §3.3.  Theorems  for  B  and  B  = 
DIBD2  and  for  graded  matrices  are  given  in  §3.4  and  in  §3.5  respectively.  We 
discuss  how  to  bound  from  below  relative  gaps,  for  example,  between  Ai  and 
A2  by  relative  gaps  between  Ai  and  A2  in  §4.  A  word  will  be  said  regarding 
Eisenstat-Ipsen’s  Theorems  in  §5.  Detailed  proofs  are  saved  for  §  §  6 ,  7,  8  and  9. 
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2  Known  Theorems 

Let  A  and  A  be  two  Hermitian  matrices  whose  eigendecompositions  are  given 
by  (1.1)  and  (1.2).  Define 


R  =  AU1-U1A1  =  (A-A)U1.  (2.1) 

In  matrix  language,  The  following  two  theorems  are  the  matrix  versions  of  two 
sin#  theorems  proved  by  Davis  and  Kahan  [2]. 

Theorem  2.1  (Davis-Kahan)  If 


1 5  =  min  I  A* 

l<i<k  ,l<j  <n  —  k 


•V k+j  |  >  0, 


then 


||  sin0(17i,  17i)||f  < 


\\R\\f 

6 


IKa-a^Wf 

6 


(2.2) 


In  this  theorem,  the  spectrum  of  Ai  and  that  of  A2  are  only  required  to  be 
disjoint.  In  the  next  theorem,  they  are  required,  more  strongly,  to  be  well- 
separated. 


Theorem  2.2  (Davis-Kahan)  Assume  there  is  an  interval  [a,  /3]  and  a  6_>  0 
such  that  the  spectrum  of  K\  lies  entirely  m  [ a,/3\  while  that  of  A2  lies  entirely 
outside  of  (a  —  6_,  [3  +  6_)  (or  such  that  the  spectrum  of  A2  lies  entirely  m  [a,/?] 
while  that  of  K\  lies  entirely  outside  of  (a  —  6_,  (3  +  6_)).  Then  for  any  unitarily 
invariant  norm  |||  •  ||| 


sin  0(17i ,  U\) 


< 


{A  -  A)U\ 

6 


(2.3) 


Now,  let  us  consider  the  perturbations  of  Singular  Value  Decompositions 
(SVD).  Let  B  and  B  be  two  m  x  n  (m  >  n)  complex  matrices  with  SVDs 


/  Si  0 

B  =  UW*  =  (lh,U2)\  0  S2 

\  0  0 

/  Si  0 

B  =  UEV*  =  (lh,U2)  \  0  S2 

Vo  0 


Vi* 

V2* 

V( 

v2* 


(2.4) 


(2.5) 


where  IJ,  IJ  <E  Um,  V,  V  £  4Jn, 


u  1,  Ui  e  cmxfc,  Vi,  Vi  e  cnxk 


(1  <  k  <  n)  and 


Si  =  diag(<7i,  •  •  • ,  <7fc),  E2  =  diag(crfc+i,  •  •  • ,  crn),  (2.6) 

Ei  =  diag(?i,  ••  •,?!,),  E2  =  diag((Tj,+i,  •  •  • ,  crn).  (2.7) 
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Define  residuals 


Rd=  BVl-UlYll  =  {B-B)Vl  and  5  =f  B*Uk -ViEi  =  (B* -B*)^.  (2.8) 
Wedin  [10]  showed  the  following  two  theorems: 

Theorem  2.3  (Wedin)  If 


c  def 

o  =  mm 


mm 

l<i<k  ,l<j  <n  —  k 


\cri  -  crk+j  |,  min  <7;  }  >  0, 
l<i<k  J 


the 


sin0(17i,  17i)|||i  +  ||  sin0(Vi,  Vi)|||, 


(2.9) 


< 


vm\2F  +  \\s\\2F  _  vik5-wiif  +  ii(5*  -B*)ih\\i 


Similarly  to  above  Davis-Kahan  theorems,  by  imposing  further  restrictions,  we 
will  have 

Theorem  2.4  (Wedin)  If  there  exist  a  >  0  and  6_  >  0  such  that 
min  <7;  >  a  +  6  and  max  fft+i  <  a, 

1  <i<k  ~  ~  1  <j<n-k  J  ~ 

then  for  any  unitarily  invariant  norm  |||  •  ||| 

max |  sin0(i/i,  U\)  ,  sin  0(Vi ,  Vi)  j  (2-10) 

<  max {|||f?||| ,  |||5I|||}  _  max{|||(^-^)T/i||  ,  |||(5*  -  B*)^  |||} 
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3  Statement  of  Theorems 

3.1  Relative  Gap  Functions 

For  two  square  diagonal  matrices 

D  =  diag(c?i,  d\,  ■  ■  -  ,dk),  D  =  diag(rfi,  d1;  ■  ■  -  ,di) 
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\\D  ^2  1  =  “in  \di\>P>a>  max  \dj\  =  ||D||2 

1 <i<k 

holds,  and  RelGap  (Hi,  £)2;  a, /?)  and  RelGap(D,  D]  a,  f3)  are  defined  to  be 

RelGap  JJ,  D;  a,  j3)  d=  RelDistp(a,  j3)  =  -^==^=, 

RelGap (D,D;a,/3)  =f  Reinistio  ft)  =  ^ 

-  VaP 

We  caution  the  reader  to  notice  the  inherent  difference  imposed  on  the  sepa¬ 
ration  between  D  and  D  associated  with  the  four  above  defined  relative  gap 
functions  RelGap2,  RelGap,  RelGap^  and  RelGap.  It  can  be  proved  that 

Proposition  3.1  If  RelGap^(Di ,  D2;  a,  /?)  and  RelGap(D,  D]  a,  f3)  exist,  then 

min  RelDistp(dj',  df)  >  RelDistp(a, /?), 

min  RelDist(dj',  df)  >  RelDist(a, /?). 

Using  the  fact  that  RelDistp(£,  £)  <  2_1/,p  RelDist(^,  Q  for  £,  (  £  C,  one  can 
show 

Proposition  3.2 

RelGap2(D,  D;  m)  <  — r=RelGap(D,  D;  m), 

v  2 

RelGap p(D,  D;a,  /3)  <  ^  RelGap(_D,  D ;  a,  /?), 

provided  RelGapp(D,  D ;  a,  (I)  exists. 

3.2  Eigenspace  Variations 

Let  A  and  A  =  D*  AD  be  two  Hermitian  matrices  whose  eigendecompositions 
are  given  by  (1.1)  and  (1.2),  where  D  is  a  nonsingular  square  matrix  close  to  I. 
Theorem  3.1  below  handles  the  case  when  eigenvalues  of  A2  and  Ai  are  not  so 
well-separated,  but  disjoint. 

Theorem  3.1  7/RelGap2  =  RelGap2(Ai ,  A2)  >  0,  then 


|  sin0(Ui,  Ui)||f  < 


RelGap2 


(3.1) 
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By  imposing  a  stronger  condition  on  the  separation  between  the  spectra  of 
A2  and  Ai ,  we  have  the  following  bound  on  any  unitarily  invariant  norm  of 
sin0(17i,  Ui). 


Theorem  3.2  Assume  that  there  exist  a  >  0  and  8  >  0  such  that  the  spectrum 
of  Ai  lies  entirely  in  [—a,a\  while  that  of  A2  lies  entirely  outside  (— ex  —  8,  a-\-8) 
(  or  such  that  the  spectrum  of  Ai  lies  entirely  outside  (—a  —  8,a-\-8)  while  that 
0/A2  lies  entirely  m  \—a,a\).  Then  for  any  unitarily  invariant  norm  |||  •  ||| 

\t(I-D*)Ui\\\q 

-p 


sin0(17i,  Ui) 


< 


f/Kl  -  D-^ 


RelGap 


where  RelGap^  =  RelGap^(Ai,  A2;  a,  a  +  8). 

3.3  Eigenspace  Variations  for  Graded  Matrices 

Consider  a  graded  Hermitian  matrix  H  =  D*  AD  £  Cnxn ,  where  A  is  an  n  x  n 
positive  definite  matrix.  (Thus  H  is  nonnegative  definite.)  Perturb  H  to  H  = 
D*  AD  in  such  a  way  that 

||A-1||2||AA||2<1, 

where 

AA  d=  A  -  A. 

The  question  we  want  to  solve  is  by  how  much  eigenspaces  of  H  are  changed. 
Write  the  eigendecompositions  of  H  and  H  as 

H  =  UAU*  =  (UUU2)  (  Al  a2  )  (  u\)  >  (3'3) 

H  =  UAU*  =  (UUU2)  ^  Al  X2  )  (  i  )  ’  (3'4) 

where  U,  U  £  UIn,  U\,  U\  £  Cnxk  (1  <  k  <  n)  and  Aj’s  and  Aj ’s  are  as  defined 

in  (1.3)  and  (1.4).  In  the  present  case,  all  A ;  >  0  and  all  Aj  >  0.  We  will  prove 

Theorem  3.3  Assume  1 1 At- 1 1 1 2 1 1 Axl| 1 2  <  1.  If  RelGap  =  RelGap(Ai,  A2)  >  0, 
then 


||  sin0(Gi,  Ui)\\f 

||(7  +  A-1/2(AA)A-1/2)1/2-(7  +  A-1/2(AA)A-1/2)-1/2||f 

<  ^  (3-5) 

RelGap 

<  ll^lk  I|AA||f 

\/l  —  1 1 At  1 1|2|| AA||2  RelGap 
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Theorem  3.4  Assume  ||A  1 1 1 2 1 1 A^4| 1 2  <  1,  and  assume  that  there  exist  a  >  0 
and  8  >  0  such  that 

max  A i  <  a  and  min  Xk+j  >  a  +  6 

1  <i<k  ~  1  <j<n-k  J  ~ 

or 

min  Xi  >  a  +  6  and  max  A k+j  <  a. 

1  <i<k  ~  1  <j<n-k  J  ~ 

Then  for  any  unitarily  invariant  norm  |||  •  ||| 
sin  0(i7i ,  Ui) 

|||(7  +  A-1/2(AA)A-1/2)1/2  -  (I  +  T-1/2(AT)T-1/2)-1/2||| 

<  ^  (3.6) 

RelGap 

<  III  AT  j 

\/l  —  RelGap 

where  RelGap  =  RelGap(Ai ,  A2;  a,  a  +  8). 

3.4  Singular  Space  Variations 

Let  B  and  B  =  D\BD2  be  two  mxn  (complex)  matrices  whose  SVDs  are  given 
by  (2.4)  and  (2.5),  where  m  >  n,  D\  and  D2  are  two  nonsingular  matrices  close 
to  identities.  The  following  two  theorems  concern  singular  space  perturbations. 

Theorem  3.5  7/RelGap2  =  RelGap2(Si ,  m)  >  0,  then 


sin  Q(U  i,Ui)\\p 


|  sin 0(Vi ,  Ri)||!' 


y/\\ (I  ~  D*i)Ui\\2F  +  \\(I  ~  DiV)Ui\\2F  +  \\(I  ~  D*2)Vi\\2F  +  II (I  ~  DfV)Vi\ 


< 


RelGap2 


(3.7) 


Similarly  to  Theorem  3.2,  by  imposing  a  stronger  condition  on  the  separation 
between  the  spectra  of  S2  and  Si ,  we  have  the  following  bound  on  any  unitarily 
invariant  norm  of  sin  Q(Ui,Ui)  and  sin  0  ( Vi ,  Vi ) . 


Theorem  3.6  If  there  exist  a  >  0  and  8  >  0  such  that 


min  <Ti  >  a  +  8  and  max  (Jk+j  <  a, 

1  <i<k  ~  1  <j<n-k  J  ~ 
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then  for  any  unitarily  invariant  norm 


RelGap 

- p 


(3.9) 

where  RelGap^  =  RelGap^(£i ,  £ 2;  a,  a  +  6). 

In  both  Theorems  3.5  and  3.6,  we  assumed  that  both  D\  and  D2  are  close  to 
identity  matrices.  But,  intuitively  D2  should  not  affect  IJ\  much  as  long  as  it  is 
close  to  a  unitary  matrix.  The  most  extremal  case  is  when  D2  E  Gn.  In  what 
follows,  we  are  going  to  explore  this  intuition.  We  will  prove 

Theorem  3.7  If 2 


RelGap2  =  RelGap2(£i ,  £2;  m)  >  — -=  max{||HC  —  D1 

2v2 


I  D*2-Df 


(3.10) 
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where  RelGap  =  RelGap(£i,  £2;  m)  an 

<i  =  \m-DpH2, 


01=1-\\D’2-D2%, 

„  =  ^m-o;%. 


The  inequality  (3.11)  clearly  says  that  D2  contributes  to  sin  0(Gi ,  Gi)  with  its 
departure  from  some  unitary  matrix,  and  similar  for  (3.12). 

Remark.  When  one  of  the  D\  and  D2  is  I,  the  assumption  (3.10)  becomes 
unnecessary  in  order  for  the  conclusion  of  Theorem  3.7  to  be  true.  In  fact,  if 
either  D\  or  D2  is  I,  we  only  need  to  assume  RelGap2(£i,  £2;  m)  >  0  which  in 
turn  insure  RelGap(£i,  S2;  m)  >  0. 

Theorem  3.8  Suppose  that  there  exist  a  >  0  and  8  >  0  such  that 
min  <Ti  >  a  +  8  and  max  <  a. 

1  <i<k  ~  1  <j<n-k  J  ~ 

Assume  that 3 

RelDistp(a,  a  +  8)  >  max{\\Dl  -  D^%,  \\D*-D^\\2}.  (3.13) 

Then  for  any  unitarily  invariant  norm  |||  •  ||| 


sin0(Gi,  Ui) 


sin0(Vi,  Vi) 


< 


j/\\\(I-  J1-1)^i|ir  +  Ilia-  D\)U~\ 

RelGap  —  eo 

- -p 


Dl  -  Df1 


< 


RelGap  —  e\ 

_ 05-14) 

\D\  —  Df1 1  y|||(7-772-1)I/i||f +  |||(7-^)I/ir 


2  RelGap  —  62 


RelGap^  — 


(3.15) 


where  RelGap^  =  RelGap^(Ei,  S2;  a,  a+8),  RelGap  =  RelGap(£i,  S2;  a,  a+8), 
and 


1 


<T  = 


2  !  +  !//> 


Id:  -  d 


Df1 1|2, 

£2  =  l||77;-772-1||2, 

Df1 1|2, 

=  op  11 

3  This  implies 


RelDist(a',  a  +  8)  >  —  max{||Z)*  —  D1  1 1 1 2 ?  H-D2  —  ^2  1  lb}* 
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Remark.  If  either  D\  or  D2  is  7,  we  only  need  to  assume  RelGap2(£i,  £2;  m)  > 

0  which  in  turn  insure  RelGap(£i,  S2;  m)  >  0,  instead  of  (3.13). 

Now,  Let’s  briefly  mention  a  possible  application  of  Theorems  3.5,  3.6,  3.7 
and  3.8.  It  has  something  to  do  with  deflation  in  computing  the  singular  value 
systems  of  a  bidiagonal  matrix.  Taking  account  of  the  remarks  we  have  made, 
we  get 


Corollary  3.1  Assume  D\  =  I  and  D2  takes  the  form 


D2 


I  X 
I 


where  X  is  a  matrix  of  suitable  dimensions.  7/RelGap2  =  RelGap2(£i,  S2;  m)  > 
0,  then 


sin0(Gi,  U\) 


< 


Q(Ui,  U\) 


2 

F 


sin0(Vi,  Vi) 


2 

F 


< 


where  RelGap  =  RelGap(£i ,  S2;  m). 

Corollary  3.2  Assume  D2  =  I  and  D\  takes  the  form 


\/2  RelGap 
RelGap2  ’ 


Di 


I  X 
I 


where  X  is  a  matrix  of  suitable  dimensions.  7/RelGap2  =  RelGap2(£i,  S2;  m)  > 
0,  then 


sin0(Vi,  Vi) 


< 


0(Gi,  U\) 


2 

F 


sin0(Vi,  Vi) 


2 

F 


< 


where  RelGap  =  RelGap(£i ,  S2;  m). 

Corollary  3.3  Assume  D\  =  7  and  D2  takes  the  form 


\/2  RelGap 

V2||X||^ 

RelGap2  ’ 


D2 


I  X 
I 


where  X  is  a  matrix  of  suitable  dimensions.  Suppose  that  there  exist  a  >  0  and 
8  >  0  such  that 


min  <Ti  >  a  +  6  and  max  (Jk+j  <  ex. 

1  <i<k  ~  1  <j<n-k  J  ~ 
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Then 


sin0(i/i,  U\) 


sin0(Vi,  Vi) 


<  J5 , 

RelGap 

<  2i/3|mi 
—  RelGap^  ’ 


where  RelGap^  =  RelGap^(£i,  £2;  a,  a-\-8),  RelGap  =  RelGap(£i,  £2;  a,  a+8). 
Corollary  3.4  Assume  D2  =  I  and  D\  takes  the  form 


Di  = 


I  X 
I 


where  X  is  a  matrix  of  suitable  dimensions.  Suppose  that  there  exist  a  >  0  ane 
8  >  0  such  that 


min  <Ti  >  a  +  6  and  max  (Jk+j  <  ex. 

1  <i<k  ~  1  <j<n-k  J  ~ 


Then 


sin0(Gi,  IJ\) 
sin0(Vi,  Vi) 


< 


< 


21/g  lll^lll 

RelGap  ’ 

PI  ' 

RelGap 


where  RelGap^  =  RelGap^(£i,  £2;  a,  a-\~8),  RelGap  =  RelGap(£i,  £2;  a,  a+i 5). 


3.5  Singular  Space  Variations  for  Graded  Matrices 

Now,  we  consider  perturbations  of  a  graded  singular  value  problem:  G  =  BD  E 
<Cnxn  and  G  =  BD  E  Cnxn,  where  it  is  assumed  that  B  is  nonsingular  and  D 
is  an  n  x  n  matrix  close  to  I.  Set 


<4  pf  '  ’ 

A  B  =  B-B. 


If  ||(AB)B  1 1 1 2  <  1,  then  B  =  B  +  A B  =  [I  +  (A B)B  1]B  is  nonsingular  also. 
Let  G  and  G  having  the  following  SVDs: 


G  =  CTCV  =  (*,%)(*  S2  )  (  {£ 

c  =  ?sw  =  (t\ty(E'  £J(| 


(3.16) 

(3.17) 
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where  U,  V,  U,  V  £  4Tn,  IJ\,  V\,  IJ\,  If  £  Cnxfc  and  Ufs  and  Uj ’s  are  given  as 
in  (2.6)  and  (2.7).  Write 

G  =  BD  =  [I  +  (A  B)B~1]BD. 

So  applying  Theorems  3.5,  3.6,  3.7  and  3.8  to  G  and  G  with  D\  =  I  +  (A B)B~1 
and  D2  =  I  gives  the  following  two  theorems  on  graded  singular  value  problems. 

Theorem  3.9  Assume  ||(AB).B_1 1|2  <  1.  7/RelGap2  =  RelGap2(£i ,  £2)  >  0, 
then 


sin0(Gi,  IJ\)  +  sin0(Vi,Vi) 


< 


\/\\(AB)B~1Ui\\p  +  ||  [T  +  B~*  (AB)*]-1  B~*  (AB)*  Ui\ 


RelGap2 


(3.18) 


<  iib-1ii21/i  + 


sin  0(Vi,  Vi) 


1 


IASI 


(l-||B-i||2||AB||2)2  RelGap2  ’ 


< 


< 


\I  +  (AB)B~1  —  (I  +  (AB)B~1)~*\\f 


(3.19) 


<  1 


2  RelGap(Ei ,  S2) 

(\\(AB)B-i  +  B-*(ABy\\F  ,  ||(AB)B-1||2  ^  ||(A B)B~'\\F 

V 

1 


1-||(AB)B  Mb/  2ReiGap(Ei,S;2) 


2/  2  RelGap(Ei ,  S2) 

Theorem  3.10  Assume  ||(AB)B_1||2  <  1.  If  there  exist  a  >  0  and  8  >  0  such 
that 

min  <Ti  >  a  +  6  and  max  <?*,+,'  <  a 

1  <i<k  ~  1  <j<n-k  J  ~ 

or,  the  other  way  around,  i.e., 

max  Ci  <  a  and  min  (fk+i  >  a  +  6, 

1 <i<k  ~  1 <j<n-k  J  ~ 


then  for  any  unitarily  invariant  norm  |||  •  ||| 
max|  sin0(Gi,  U\)  ,  sin0(Vi,Vi)  j 


<  max{|||(AR)R-1G1|||  ,  |||  [J  +  B~*  (AB)*]-1  B~*  (AByiJ,  ||| }  2Q 


< 


II B- 


RelGap0 

IIASIII 


1- ||B-i||2||AB||2  RelGapc 
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sin Q(Ui,  Ui) 


sin0(Vi,  Vi) 


< 


^\i(AB)B-1U1\iq  +  |||[/  +  B~*  (AB)*]-1  B~*  (AB)*Ui\ 


RelGap 


(3.21) 


<  \\B-1\\2di  + 


1 


HASH 


v  (1-||B-1||2||AB||2)«  RelGapp’ 

sin0(Vi,  Vi) 

mI+(AB)B~1  -  (I  +  (AB)B~1)~1 1 


< 


< 


2  RelGap(£i ,  S2) 

\(AB)B~1  +  B-*(AB)*\\\  ||(AB)B-1||2  \  |||(AB)B-1| 


(3.22) 


<  1 


|||(AB)B-1| 

1 


1  -  ||(A5)5  1 1|2 y  2RdGlp(Si,S2) 
Is-MhlllABlll 


2/  2  RelGap(Si,  S2) 


where  RelGap^  =  RelGap^(Ei ,  S2;  a,  a  +  8),  ( and  thus  RelGap^  =  8 / (a  +  8) ). 

Remark.  The  inequalities  (3.19)  and  (3.22)  may  provide  much  tighter  bounds 
than  (3.18)  and  (3.20),  especially  when  I  +  ( AB)B~ 1  is  very  close  to  a  skew 
Hermitian  matrix. 
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4  More  on  Relative  Gaps 


In  theorems  of  §3,  relative  gaps  RelGap2,  RelGap,  RelGap^  and  RelGap  play 

an  indispensable  role.  Those  gaps  are  imposed  on  either  between  Ai  and  A2  or 
between  Si  and  S2.  In  some  applications,  it  may  be  more  convenient  to  have 
theorems  where  only  positive  relative  gaps  between  Ai  and  A2  or  between  Si 
and  S2  are  assumed.  Based  on  results  of  Ostrowski  [5,  pp.  224-225],  Barlow 
and  Demmel  [1],  Demmel  and  Veselic  [3],  Mathias  [8],  and  Li  [7],  theorems  in  §3 
can  be  modified  to  accommodate  this  need.  In  what  follows,  we  list  inequalities 
for  how  to  bound  relative  gaps  between  Ai  and  A2  or  between  Si  and  S2  from 
below  for  each  theorem  by  their  corresponding  relative  gaps  between  Ai  and  A2 
or  between  Si  and  S2.  The  derivations  of  these  inequalities  depends  on  the  fact 
that  [7] 

1.  RelDisti,  RelDist2  and  RelDistoo  are  metrics  on  M; 

2.  RelDistp  is  a  metric  on  M>o;  and 

3.  RelDist  is  a  generalized  metric  on  M>o- 

For  Theorem  3.1:  If  ||  J  —  D*  D\\2  <  RelGap2(Ai,  A2), 

RelGap2(Ai,  A2)  >  RelGap2(Ai ,  A2)  -  \\I-  D*D\\2. 


For  Theorem  3.2:  Assume  that  for  some  0  <  oq  <  /?i ,  RelGap^ (Ai ,  A2;  a\,  f3\) 

is  dehnable.  If  ||  J— D*  D\\2  <  RelGap  (Ai,  A2;  oq,  (3\),  then  RelGap^ (Ai ,  A 2;a,  f3) 
is  dehnable  for  some  0  <  a  <  [3  and 

RelGap^ (Ai ,  A 2;a,  f3)  >  RelGap^ (Ai ,  A2;  oq,  /3i)  —  ||  J  —  D* D\\2. 


For  Theorem  3.3:  If  e  <  RelGap(Ai ,  A2), 

RelGap(Ai,  A2)  >  RelGap(Ai,  A2)  —  e, 


where  e  = 


HA-MhllAAIh 

Vl-||A-M|a||AA||a' 


For  Theorem  3.4:  Assume  that  for  some  0  <  u\  <  f3\,  RelGap(Ai,  A2;  ai,/?i) 

is  dehnable.  If  e  <  RelGap(Ai,  A2;  a\,  /?i ) ,  then  RelGap(Ai ,  A2;  a,  f3)  is 
dehnable  for  some  0  <  a  <  [3  and 


RelGap(Ai,  A2;  a,  f3)  >  RelGap(Ai,  A2;  oq,  /?i )  —  e, 


where  e  = 


||A-1||3||AA||3 

Vi-||A-1||2||AA||2- 
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For  Theorem  3.5:  Let 


£ —  max{  1 1  C^min  (-^1  )crmin  (-^2)  1 7  |1  ^max (7^1  )^max (-^2 )  |  }  • 

If  e  <  RelGap2(Si,  S2;  m),  then 

RelGap2(£i ,  S2;  m)  >  RelGap2(Si ,  S2;  m)  —  e. 

For  Theorems  3.6:  Let 

£  —  niax{  1 1  C^min  (T-h  )(7min  (Z72)  | ,  |1  ^max  (-^1  )^max  (772  )  |  }  • 

Assume  that  for  some  0  <  u\  <  f3\,  RelGap^(Si,  S2;  a\,  /?i )  is  definable. 

If  e  <  RelGap^(£i ,  S2;  a\,  /?i )  then  RelGap^(Si,  S2;  a,  f3)  is  definable  for 
some  0  <  a  <  [3  and 

RelGap^(£i,  S2;  a,  f3)  >  RelGap^(Si,  S2;  a\,  /?i )  —  e. 

For  Theorems  3.7:  Let  e\,  e2,  ?i  and  ?2  be  as  defined  in  the  theorem.  If 
G  +  G  +  max{?i,  ?2}  <  RelGap(Si ,  S2;  m), 

then 

RelGap2(£i,  S2;  m)  >  RelGap2(Si ,  S2;  m)  —  t\  —  e2, 
RelGap(Ei,  S2;  m)  >  RelGap(Si,  S2;  m)  —  7\  —  ?2. 

For  Theorems  3.8:  Let  ei ,  £2,  G  and  ?2  be  as  defined  in  the  theorem.  Assume 
that  for  some  0  <  u\  <  f3\,  RelGap(Si ,  S2;  a\,  /?i )  is  definable.  If 

G  +  G  +  maxjfi ,  £2}  <  RelGap(£i,  S2 ;  »i ,  (3\), 

then  RelGap(Ei,  S2;  a,  f3)  is  dehnable  for  some  0  <  a  <  [3  and 

RelGap^(£i,  S2;  a,  f3)  >  RelGap  (Si,  S2;  —  ei  —  e2, 

RelGap(Si,  S2;  a,  f3)  >  RelGap(Si ,  S2;  a\,  /?i )  —  7\  —  ?2. 

For  Theorem  3.9:  Let  e  =  ||B_1||2||AB||2  and 

1  \  ||5-1||2||A5||2 

l-||B-i||2||AB||2y  2 

If  e  <  RelGap2(Si,  S2)  and,  7  <  RelGap(Si,  S2), 

RelGap2(Si ,  S2)  >  RelGap2(Si,  S2)  —  e, 

RelGap(Si ,  S2)  >  RelGap(Si ,  S2)  —  7. 
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For  Theorem  3.10:  Let  e=  \\B  1 1 1 2 1 1 A ^ 1 1 2  and 

~  (■,  ,  1  \  II-^-1I|2||a_b||2 

^  +l-||5-i||2||A5||2y  2 

Assume  that  for  some  0  <  u\  <  f3\,  RelGap  (Si,  S2;  ai,/?i)  and 

RelGap(Si,  S2;  are  definable.  If  e  <  RelGap  (Si,  S2;  ai, /?i)  and 

7  <  RelGap(Si,  S2;  a\,  /?i ) ,  then  RelGap^(Si,  S2;  a,  f3)  and  RelGap(Si,  S2;  a,  f3) 
are  definable  for  some  0  <  a  <  [3  and 

RelGap^(Si ,  S2;  a,  f3)  >  RelGap  (Si,  S2;  ai, /?i)  —  e, 

RelGap(Si ,  S2;  a,  f3)  >  RelGap(Si,  S2;  a\,  /?i)  —  7. 
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5  A  Word  on  Eisenstat-Ipsen’s  Theorems 


Eisenstat  and  Ipsen  [4]  have  obtained  a  few  bounds  on  eigenspace  variations  for 
A  and  A  =  D* AD  and  on  singular  space  variations  for  B  and  =  B  =  D\BD2- 
Here  we  are  not  going  to  state  their  results  for  subspaces  of  dimension  k  >  1 
since  their  results  contain  a  factor  of  \/k  which  makes  their  results  uncompetitive 
to  ours. 

Regarding  to  the  problem  related  to  Theorem  3.1,  Eisenstat  and  Ipsen  [4] 
tried  to  bound  the  angle  Oj  between  uj  (1  <  j  <  k )  and  the  IZ(Ui),  where 
«i,  u2,  •  •  • ,  wj;  are  the  columns  of  U\.  They  showed 


where 


sin  Oj  < 


\\I 


D-*D-% 

Vj 


\\I~D  ||2, 


def 
7]j  = 


min 

k  + 1  <  i  <  n 
OC 


|a,-a7| 

|A,| 


if  A  j  0, 

otherwise. 


(5.1) 


The  inequality  (5.1)  does  provide  a  nice  bound.  Generally, 


sin  Oj  <  ||  sin0(G1 ,  Ui)\\2 


and  all  of  them  may  be  strict.  To  make  a  fair  comparison  to  our  inequality 
(3.1),  let’s  consider  the  case  k  =  1.  One  infers  from  our  inequality  (3.1)  that4 


I-D-'Wt  +  \\I-D*\\l 

min  RelDist2(A8' ,  Ai) 

2  <i<n 


sin  0\  < 


(5.2) 


At  the  first  sight,  one  might  think  (5.1)  is  potentially  sharper  because  rj i  may 
be  much  larger  than  min  RelDist2(A8,  Ai).  This  is  totally  wrong!  Yes,  rft  may 

2  <i<n 

be  very  large,  but  this  is  shadowed  by  the  extra  term  ||J  —  D 1 1 2  in  (5.1)  which 
stays  no  matter  how  large  rft  is.  As  a  matter  of  fact5,  we  should  really  think  of 

4By  treating  A  and  A  symmetrically,  one  can  see  Theorem  3.1  remains  valid  with 
RelGap2  ( Ai ,  A2 )  replaced  by  RelGap2  ( Ai ,  A2 ) . 

5  Going  through  our  proof  of  Theorem  3.1,  we  can  see  that  when  k  =  1  and  Ai  =  0  we  have 

sin#i  <  ||7  —  ZA 1 12 - 

In  fact,  besides  (6.1)  one  also  has 

Ai U*U2  -  U*U2A2  =  MU^D-1  -  I)U2  +  U*(I-  D*)U2A2. 

When  Ai  =  0,  this  equation  reduces  to 

-u*u2  =  U*(I-D*)U2 , 

provided  Xj  ^  0  for  j  =  k  +  1,  •  •  •  ,  n. 


19 


(5.1)  as  (at  least  for  tiny  ||J  —  D\\ 2) 


\\I-D-*D-%  +  \\I-D\\2 

min  RelDistoo(A2-,  Ai) 

2  <i<n 


(5.3) 


Well,  still  we  cannot  say  definitely  which  one  of  (5.2)  and  (5.3)  is  always  sharper 
than  the  other.  However  asymptotically,  (5.3)  is  less  sharp  by  a  factor  3/2.  In 
fact,  if  I  —  D  is  very  tiny,  then 

min  RelDist2(A8- ,  Ai)  ps  — —  min  RelDist00(AJ  ,  Ai), 

2 < 2 < n  a/ 2  2 < i < n 

^\\I-D-mi  +  \\I-D*\\l  «  V2\\I-D\\2, 

\\I-D-*D-%  +  \\I-D\\2  «  3||/-£>||2. 

So  asymptotically,  the  inequalities  (5.2)  and  (5.3)  read,  respectively, 

sin!?!  <  - 2||7~J|12  ~  +0(\\I-D\\l), 

~  min  RelDist00(Ai,  Ai) 

2  <i<n 

sin  I?!  <  - 3||7~J|12  ~  +  0(\\I  —  D\\l). 

min  RelDist00(Ai,  Ai) 

2  <i<n 


Eisenstat  and  Ipsen  [4]  treated  singular  value  problems  in  a  very  similar  way. 
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6  Proofs  of  Theorems  3.1  and  3.2 

Let  R  be  as  defined  in  (2.1).  Notice 

U*R  =  UfAIh  -  U%U1A1 

=  l2u;ul  -  pup, 

U*2R  =  U*2{A-A)Ui 

=  U*  [A- D*A  + D*A- D*AD\U1 
=  U2  [(/  —  D*  ) A  +  D*AD(D~1  -  /)]  Ui 
=  A2Ef2*  (IT1  -I)  Ul  +u;(l  -D*)lhM 

Thus,  we  have 

A2U2Ui  -  U2U1A1  =  A2U2  (D-1  -  Ip!  +  U2  (I  —  D*)UiAi.  (6.1) 


Proof  of  Theorem  3.1:  Set 

Q  =  UfU1  =  (qlJ), 

E  =  UfiD-1  -I)U1  =  (elJ), 
E  =  U*{I-D*)U1  =  {Tij). 


Then  the  equation  (6.1)  reads  A2 Q  —  QE\  =  K2E  +  E Ai,  or  componentwisely 
^ k-\-iQij  Ej^j  —  ^k-\-i^ij  T  ,  SO 


l(Afc+i  -  Aj)qij  | 2  <  (A|+i  +  A|)(|es'j|2  +  \eij  |2), 
which  yields,  under  the  conditions  of  Theorem  3.1,  that 

,  ,2  <  NP  +  IEf  <  NP  +  IEf. 

~  RelDist^Afc+i,  Xj)  ~  RelGap2 


So 


i  —  k  k 


n-k  k  E  E(M2  +  M2) 

IMI^  =  S§l9iJ’|2“  8  ^  RelGap2 

I \U*2(D~l  -  I)U\  ||^  +  ||  U*2(I  -  DpMl 


< 


1 


RelGap2 

(Up-1  -phWlT  11(7- D*)Ui\\2f)- 


RelGap2 

This  completes  the  proof  of  Theorem  3.1,  since  ||  sin  0(Gi ,  O))^  =  ||Q||f- 
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Proof  of  Theorem  3.2:  It  follows  from  the  perturbation  equation  (6.1)  that 


UfUi  -  AfLUfU1A1  =  Uf  (D~l  —  T)U\  +  -  D*)Uiki. 


,-ii 


By  the  assumptions  of  the  theorem,  we  know  that  ||Ai||2  <  aand||A2  \\2 
Therefore 


1  - 

< 

< 

< 

< 


ex  8 
U;Ue 
Uflh 
Uflh 


u;ue 


1 


IJ^Ue 

UfUe 


ex  6 

~\fc% 

kpUphP 
u;u!  -  kpUphP 
UfiD-1  -  Ipe  +  kf  lUf  (I  -D* )UiAi 

I  —  D*)IJ  1 1 


< 


1  + 


aP 


(by  (6.2)) 


(a  +  6)p 

from  which  the  inequality  (3.2)  follows. 


ot  b 

i/uD-i-mr  +  ui-Dppr 


(6.2) 

1 

a  +  b  ' 
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7  Proofs  of  Theorems  3.5  and  3.6 

Let  R  and  S  be  defined  as  in  (2.8). 

7.1  The  Square  Case:  m  =  n 

When  m  =  n,  the  SVDs  (2.4)  and  (2.5)  read 

B  =  VT.V'  =  (lh,lh)(*'  Ej)(!|). 

S  =  rav*  =  (£?,. if.) (£l  g3)(!) 

Then 

u;r  =  u2bvi  —  u^Ui'Ei 

U^R  =  U;{B-B)Vi  =  U;{D*1BD2-D*1B  +  D*1B-B)Vi 
=  UZ  [b(I  -  D21)  +  (Dt  -I)b]  W 
=  +UZ(D1  -/)th£i. 

On  the  other  hand,  for  the  residual  S,  we  have 

V2S  =  V2B*Ui  -  C2*Vi£i, 

=  F2U2U\  —  V^Vi'Ei. 

V*S  =  V2(B*  -  B*)Ui  =  V2(D*2B*D1  -  D*2B*  +  D*2B*  -  B*)Ui 

=  V2  \B*  (I  -  D-[v)  +  (Dl  -7)5*]  Ui 

=  f2u*2{i  -  d^1)u1  +  v2*(d;  -j)Ti£i. 

Therefore,  we  obtained  two  perturbation  equations  as  follows. 

S2T2*Ti-Cf2*tfi£i  =  Z2V2*(I  -  D^jVi  +  U^Dl  -1)11^,  (7.1) 
F2U2U\  —  V2ViT,i  =  F2U2  (I  —  D^1)U\  +  V2  (D2  —  I)ViY<i.  (7.2) 

Set 

Q  =  UJ  U!  =  (<Zy ) ,  Q  =  VJVX=  (qtJ ) , 

E  =  VJ(I-D21)Vl  =  (elj),  E=Uf(D*1-I)U1  =(£„),  (7.3) 

F  =  U2(I  -  D^1)IJ\  =  (/,-,■),  F  =  V2*(D*2  -  /)Ti  =  (/,-> )• 
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It  is  easy  to  see  that  for  any  unitarily  invariant  norm  |||  •  |||  (ref.  (1.5)) 


sin©(t/!,  C/r)  =  IQI  , 

sin 

'0“ 

ii 

(7.4) 

iii  di  <  ||a  -Df1)^  ||, 

E 

<  111(^-7)01, 

(7.5) 

llldll<  Ilia- r»i_1)Ci in, 

d 

<  11(77; -7)0  |||. 

(7.6) 

Proof  of  Theorem  3.5:  Componentwisely,  with  the  assignments  of  (7.3),  the 
equations  (7.1)  and  (7.2)  read 


Ck-\-iQij  Qij  &  j  —  Gk+i^ij  \  &ij  &  j  5 

Ck+iQij  Qij  &  j  —  &k-\-ifij 

where  1  <  i  <  n  —  k,  1  <  j  <  k,  which  yields 

I Vk+Wj  -  qijCTj  I2  <  {v'l+i  T  <Tj)(\eij  I2  +  I eij  I2) 

=>  \etJ\2  +  \TtJ\2>  ~  W* 


~9  .  9  i 

at+t  +  <7 


K +*'  Qij  -  Qij  CTj  1 2  <  ( crl +i  +  a] )  ( |  f{j  | 2  +  |  /„• , 


1 2  |  if  1 2  \  I  Vk+iQij  lijVj  I 


I/d  +  l/d  > 


al+i  +  a] 


Summing  on  i  and  j  for  i  =  1,2,  •  •  •  ,n  —  k  and  for  j  =  1,2 ,  •  •  •  ,k,  we  get 

| Ck+iQij  Qij^j  I  d" 


iw+iw+imiF+iw>EE- 

i  =  1  J  =  1 


dd  +  <7? 


Since 


TjO'j I  d-  7 i j  TjO'j I 

=  ^fc+ikd  +  l?d  17 j  20fJ:(cTfc  _|_J  (Ty  ) 

■©fc+i  k*i  I  +  l?d  17 j  29?‘((T* _|_J  gz-y  (Ty  ) 

>  (ak+i  -  <ri)2(kii|2  +  Iftil2), 


where  -K(-)  takes  the  real  part  of  a  complex  number,  so 


iidd  +  iidd  +  iidd  +  iidd 


n  —  k  k  n2 

>  ZE(2*:y 


i=i  j= i 


ak+i  +  <7 


l?d2) 


>  RelGap2(||0||^  +  ||0||d, 
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which,  together  with  the  equations  (7.4),  (7.5)  and  (7.6)  for  |||  •  |||  =  ||  •  ||i? , 
complete  the  proof  of  the  theorem.  I 

Proof  of  Theorem  3.6:  It  follows  from  the  equations  (7.1)  and  (7.2)  that 

-UfU,  =  Z2V2*(I  -  +  Uf(Dl  -I)U,,  (7.7) 

-V2*V,  =  Z2Uf(I  -  Df^lhZf1 +  V2*(D*  -7)1/!.  (7.8) 


To  prove  the  inequality  (3.8),  we  consider  first  the  case 


mu. 


> 

.-ii 


V2*V! 


By  the  assumptions  of  the  theorem,  we  know  that  ||£2  <  ex  and  ||S1  1 1 2  <  Opr 
Now  it  follows  from  (7.7)  that 


1  - 

< 

< 

< 

< 


ex  6 
UfU, 

u;u, 
u;u, 

Uflh 


U*2u 1 


U*2U, 

v2*v, 


1 


ex  6 

1 


ex  6 


lar'lh 


i-i 


< 

<  ex 

< 


s2i/2*i/1e^ 

% 2V2*(I  -  +  U;(Dl  -  1)11, 


Z2V2*(I  -  +  Uf{Dl-I)U, 


(by  (7.7)) 


V2*(I-Dfv)V, 


1 


u2{di  -  i)u. 


Qt  -\-  b 


Qt  b 

(I-Df1)V,\\\  +  1(77)  -I)U, I 


< 


1  + 


exP 


(a  +  6)p 


which  produces  that  if 


mu. 


+  11(01 -i)u,r 

v2*v,\ 


> 


mu. 


< 


Villa  -Df^f  +  \\(Dl-I)U,  r 


Similarly,  if 


mu. 


< 


v:v. 


RelGap^ 

one  can  prove,  using  (7.8),  that 


(7.9) 


V2*V! 


< 


^\\\(I-Dfyj,\\\q+l\(D*-I)V,r 


RelGap 


(7.10) 
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The  inequality  (3.8)  is  now  a  simple  consequence  of  (7.9),  (7.10)  and  (7.4). 

The  inequality  (3.9)  is  left  unproved.  Let  the  assignments  of  (7.3)  hold.  The 
equations  (7.1)  and  (7.2)  are  equivalent  to  the  following  single  matrix  equation 
with  dimensions  doubled. 


Under  the  conditions  of  Theorem  3.6,  it  is  easy  to  verify  that 


Applying  the  similar  trick  we  used  so  far,  we  can  get 


< 


1 

RelGap^ 


(7.11) 


Since  Q  and  sin0(Vi,Vi)  have  the  same  singular  values  and  so  do  Q  and 
sin0(Ui,  U\)  (ref.  (7.4)), 


sin0(Ui,  Ui) 

sin0(Vi,  Vi) 


Note  also 


(7.12) 


(I-DT^V, 


E 


F 


u9* 


(Dl  -  7)Ui 


(D*  -  7)Ui 
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Thus,  one  has 


(I-DT^V, 

W  -I)U i  \ 

(D*  -W  ) 


(7.13) 

(7.14) 


The  inequality  (3.9)  now  follows  from  (7.11),  (7.12),  (7.13)  and  (7.14).  I 


7.2  The  Non-Square  Case:  m  >  n 

Augment  B  and  B  by  a  zero  block  0 m,m-n  as  follows. 

71a  —  (71,  0 m,m— n)  and  Ba  —  (71,  0m  m_n). 
Since  B  =  D\BD’2,  we  get 

Ba  =  D*1Ba(D 2  \=D\BaDa2. 

V  -Lm  —  n  J 


From  the  SVDs  (2.4)  and  (2.5)  of  B  and  B ,  one  can  calculate  the  SVDs  of  Ba 
and  Ba: 

Ba  =  UT,aV:  =(UUU2)  (  El 

sa  =  u^av:  =  (uuu2)  ^ 

where 


similarly  for  £a2,  Cai  and  Va2-  The  following  fact  is  easy  to  establish 


Sa2  — 


0  m  — 


m  —  n,m  —  n 


,  14l  = 


Cl 

Om-n.fc 


,142  = 


14 


4r?,  — 


al 


a2 


^a2 


1/* 

a. 

14 


a2 


(7.15) 

(7.16) 


sin  0(14,1,  Ca) 


sin0(Vi,  Vi) 


Applying  the  square  case  of  Theorems  3.5  and  3.6  to  m  x  m  matrices  Ba  and 
Ba  defined  above  will  complete  the  proofs.  I 
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8  Proofs  of  Theorems  3.7  and  3.8 


We  have  seen  in  §7  how  to  deal  with  the  nonsquare  case  by  converting  it  to  the 
square  case.  So  here  we  will  only  give  proofs  for  the  square  case: 


m  =  n. 

Let  B  =  D\B  and  B  =  BD2  and  their  SVDs  be 

*  =  ^v-  =  iu,x y(£l  g2)(g). 

6  =  OH"  =  iuuih)  (  E‘  j2)(|). 

where  U,U  E  UJn,  V,  V  E  UIn,  Uu  Ui  E  Cnxk ,  Vi,  Vi  E  Cnxk  and 
Si  =  diag(<7i,  •  •  -  ,ak),  S2  =  diag(oq,+i,  •  •  •,<?„), 

Si  =  diag(<7i,  •  •  • ,  o-fc),  S2  =  diag(<7fc+i,  •  •  • ,  &n). 

Because  the  gap  assumptions  made  in  Theorem  3.7  and  Theorem  3.8,  we  can 
make  [7] 

RelDist(<78' ,  <7j)  <  i|| D{  -  RelDist(oy,  3y)  <  t^\\D2  ~ 

RelDist((T8',  &i)  <  i||S>2  ~  D21\U,  RelDist(<78  ,  oy)  <  )j\\D\  -  D^\\2. 

By  the  fact  RelDistp(£,  £)  <  2_1/,pRelDist(<J;,  (),  the  above  four  inequalities  im¬ 
ply 


RelDistp(<7, ■,£,■)  <  ^T7j||T»i*  -  D1  1||2, 
RelDistp((Tj' ,  &{)  <  ^Lj-Wd;  -  D^\\2, 
Consider  B  and  B  =  BD2.  We  have 


RelDistp((Tj',  cy)  <  — ^11^2 
RelDistp(dy,  3y)  <  ^Lj-WDI 


D,1 1|2, 
Di1 1|2. 


BB* 


t7SS*C*  =  (UUU2) 


Cl* 

\J*2 


BB* 


CSS*C*  =  (Ci,C2) 


UJ 

u*2 


Notice  that 

BB*  -  BB* 

U*(BB*  -  BB*)U 
U*B(D*2  -  D2V)B*U 


BD*2B*  -  BD2lB* 
SiDl-^B*, 
ss *u*u  -  u*u ss*, 

S  V*(D*2  -  D2v)V  s*. 


(8.1) 

(8.2) 
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Thus,  we  have  Y?U*U  —  U*UY?  =  T,V*(D2  —  D2  1)VT,  which  gives 
f,lU*2Ui  -  U*2Uit\  =  f,2V2{Dl  -  Df1)^ lSi. 


Consider  now  B  and  B  =  D\B.  We  have 


B*B 

B*B 


Vt*tV*  =  (Ci,c2) 
=  {Vi,V2) 


Notice  that 

B*B-B*B  =  B*  D\B  —  B*  Df1  B 
=  B*(Dl  -  Dfv)B, 

V*(B*B-B*B)V  =  Y,*Y,V*V  -V*Vt*t, 
V*B*(D*l  —  Df1)BV  =  Y?U*{D\-  D^)Ut. 

Thus,  we  have  Y?V*V  —  V*Vtf2  =  HU*(D*  —  D which  gives 
£|V"2*Vi  -  V2V! Si  =  S2 U2{Dl  - 
Two  other  useful  eigendecompostions  are 

BB*  =  U^*U*  =  (UUU2)  ^  s2  )  (  ^  )  , 

B*B  =  VX*XV*  =  (VUV2)  ^  S2 


Proof  of  Theorems  3.7:  It  follows  from  (8.3)  and  (8.6)  that 


sin0(Ci, 

||sin0(Vi, 


VfiD^-Df1)^ 

_ r_ 

RdG^p  (Ef,£2) 
\\UUDt  ~  Df1)^ 

y*  y  N _ F_ 

2  F  ~  RelGap(Ef ,  £2) 


(8.3) 

(8.4) 

(8.5) 

(8.6) 

(8.7) 

(8.8) 


On  the  other  hand,  applying  Theorem  3.1  to  BB*  and  BB*  =  D\BB* D\  leads 
to  (ref.  (8.7)  and  (8.1)) 


sinO(Ci, 


< 

F 


RelGap2(S|,  £2) 
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Applying  Theorem  3.1  to  B* B  and  B* B  =  D^B* BD2  leads  to  (ref.  (8.8)  and 

(8-4))  _ 


sin 0( Vi,  Vi) || F  < 


iIIf 


RelGap2(£f,  E|) 


Notice  that  [7] 


RdGap(Ef,E|)  > 
RelGap^,!^)  > 

RelGap^E^I^)  > 

RelGap2(Xlf ,  S2)  > 


2RelGap(Ei,S;2)  >  2  [RdGap(Ei,  S2)  -?i]  , 

2  RelGap(Si ,  S2)  >  2  [RdG^p(£i ,  S2)  -  ?2]  , 

RelGap2(Ei,  £2)  >  RelGap2(Ei,  S2)  —  e2, 
RelGap2(Ei ,  S2)  >  RelGap2(Ei ,  S2)  —  e\ , 


where 

fi  =  )lie;-Dr‘ib, 

.,  =  i45l|e;-er,|b, 


<■-'  =  jllK  -Dplk, 


The  proof  will  be  completed  by  employing  [9] 


sin0(Gi,  U\) 

< 

F 

sin0(Gi,  U\) 

+ 

F 

sin  0(Gi ,  U\) 

F 

sin  0(Vi ,  Vi) 

< 

F  1 

|sin  0(Vi ,  Vi)  || 

F 

sin0(Vi,  Vi) 

F 

Proof  of  Theorems  3.8:  Denote  [3  =  a  +  6.  Let  a  and  a  be  the  largest  positive 
numbers  so  that 

RelDist(a,a)  <  — 1|£)2  —  Df1^  and  RelDist(a,  d)  <  — 1|£)*  —  Df1^ 
which  guarantee  that 

||S2||2  <  a  and  ||S2||2  <  d, 


and 

RelDistp(a,  a)  <  ^1/p \\D*2-D^  ||2  and  RelDistp(a,  d)  <  ^1/p  ||T>) -Df1 1|2 ; 
and  let  [3  and  f3  be  the  smallest  numbers  so  that 

RelDist(/3,  /?)  <  i||D*  -  and  RelDist(/3,  (3)  <  i||.D2  - 
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which  guarantee  that 


ll^i 


>  /3  and  ||X]1 


>P 


and 

RelDistp  (/?,/?)  <  2ll1/p\\Dl-D^%  and  RelDistp(/?,  /?)  <  2l+\/p \\D*2 -D^1 1|2. 

Because  of  our  gap  assumptions,  min{/3,  /?}  >  a  and  /?  >  max{a,  d}. 

It  follows  from  (8.3)  and  (8.6)  that 


sin  Q(Ui ,  IJ\) 
sin0(Vi,  Vi) 


u;u. 


v:v. 


< 


< 


V^Dl-D^V, 


RelGap (£2,  £2;  a2, /?2) 

U?ADl-D^)lh 


RelGap (£2,  £2;  a2, /?2) 


On  the  other  hand,  applying  Theorem  3.2  to  BB*  and  SB*  =  D\BB* D,  leads 
to  (ref.  (8.7)  and  (8.1)) 


sin  Q(Ui,  IJ\) 


< 


y|||(7-Br1)G1|f  +  |||(7-^)G1|||^ 

RelGap  (£2,  £2;  S2,  (]2) 


Applying  Theorem  3.1  to  B* B  and  B* B  =  D^B* BD2  leads  to  (ref.  (8.8)  and 
(8.4)) 

I,  •  filler  -  rjj1)!",  |f  + 1||(/  -  z>3) v,  r 

sm0(\/i,  17  ^ 


< 


RelGap,(£2,£2;d2,/?2) 


Notice  that  [7] 


S2;a2,^2) 

> 

2  RelGap(£i ,  £2;  a, 

M! 

to  to 

Q 

to 

"Ccx 

to 

> 

2  RelGap(£i ,  £2;  a, 

Ei;d2,/32) 

> 

RelGap  (£1 ,  £2;  a, , 

E2;d2,/32) 

> 

RelGap  (£1 ,  £2;  d, , 

where 


^1  =  -i2, 


?2  =  -||£>2*-£>2-i||2, 


=  2ITI7?II^  -  D~^W>  £2  =  2lTi7?ll^  -  D21 1|2. 


31 


The  proof  will  be  completed  by  employing  [9] 


sin0(17i,  U\) 

< 

sin0(17i,  U\) 

+ 

sin  0(17i ,  U\) 

sin0(Vi,  Vi) 

< 

sin0(Vi,  Vi)  HI 

+ 

sin0(Vi,  Vi) 
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9  Proofs  of  Theorems  3.3  and  3.4 

Notice  that 


H  =  D*AD  =  (A1/2D)*A1/2D  =  BB*, 

H  =  D*A1/2(I  +  A-1/2(AA)A~1/2)A1/2D 


+  A~ 


=f  BB* 


where 


B  =  D*A1/ 2 

B  =  D 


1/2  d=  D*A1/2D2  =  BD2. 


Given  the  eigendecompositions  of  H  and  H  as  in  (3.3)  and  (3.4),  one  easily  see 
that  B  and  B  admit  the  following  SVDs. 


B  =  UA1/2V*  =  (UUU2)  Al 


1/2 


A. 


1/2 


B  =  UA1/2V*  =  (UUU2) 


A 


1/2 


V? 

v2* 

Vi* 


A  T  I  V  V2* 


where  [/,  U  are  the  same  as  in  (3.3)  and  (3.4),  Vi,  V\  E  Cnx  .  Notice  that 

H-H  =  BB*-BB*  =  BD*2B*  -  BDT^B* 

=  B(D*  -  Dj^B* , 

U*(H  —  H)U  =  AU*U  -  U*U  A, 

U*B(D*2  -  D2V)B*U  =  A1/2V*(D*2  -  D2v)VA1/2. 

Thus,  A U*U  -  B*B A  =  A  1!2V*{D*2  -  D21)VA1/2  which  yields 
A2G*Gi  -  U;U1A1  =  1\  /2V2*(D*2  -  D2v)ViA\12. 

The  following  inequality  will  be  very  useful  in  the  rest  of  our  proofs. 

|h* (d2  -  D21)Vi 

( I  +  A-1/2(AA)A-1/2)1/2  -  (I  +  A-1/2(AA)A~1/2)-1/2 
<  11(7  +  A-1/2(AA)A-1/2)-1/2||2  |||a-1/2(AA)A-1/2 

b 


(9.1) 


<  d*2-d^ 


< 


1 A  A 1 

V1-||A-1||2||AA||2- 
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Proof  of  Theorem  3.3:  It  follow  from  (9.1)  that 


sin0(I7i,  Ui) 


mu  i 


< 


RelGap 


as  required.  I 

Proof  of  Theorem  3.4:  It  follow  from  (9.1)  that 


sin0(I7i,  Ui) 


Vf(D*2  -  Df1)^ 
RelGap 


as  required.  I 
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